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Abstract 

Using the regenerative scheme of |3, we establish a functional central limit theorem 
OO . (FCLT) for discrete time stochastic processes (chains) with summable memory decay. 

' Furthermore, under stronger assumptions on the memory decay, we identify the limit- 

. ing variance in terms of the process only. As applications, we define classes of binary 

autoregressive processes and power-law Ising chains for which the FCLT is fulfilled. 
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■ 1 Introduction and preliminaries 

1.1 Introduction 

(N ■ 

!>■ I Chains are discrete-time stochastic processes with infinite memory that are natural extensions 

of Markov chains when the associated process depends on its whole past. Such processes have 
fSj . been extensively studied (see e.g. Fernandez and Maillard [11] and references therein), but 

(N ■ surprisingly very few is known about limit theorems. In this paper, we partially fill this gap 

by establishing a functional central limit theorem. 

' Historically, the first central limit theorems for chains have been established, under strong 

O ■ ergodic assumptions, by Cohn (1966) [7], Ibragimov and Linnik (1971) [2] (see also losifescu 

and Grigorescu (1990) tl3j). More recently, the empirical entropies of chains with exponential 
memory decay have been studied both in terms of their limit behavior (Gabrielli, Galves and 
Guiol (2003) [12]) and their large deviations (Chazottes and Gabrielli (2005) [3j). 

Limit theorems such as LCLT and LIL have been broadly studied for Markovian chains 
(see e.g. Meyn and Tweedie [l9]). Regeneration methods, introduced by Chung (1967) [5] 
and refined by Chen (1999) [1], have been used to divide the Markov chain into independent 
random blocks in order to derive such limit theorems. This constitute a motivating challenge 
to extend those techniques to the non-Markovian case. 

In that paper, we prove a general FCLT ( Theorem 1 2. ip for chains satisfying the regenerative 
scheme introduced in Comets, Fernandez and Ferrari We give an explicit expression for 
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the associated limiting variance depending both on the original and regenerative processes. 
As a corollary (Corollary 12. 2|) . we give a more tractable condition on the memory decay of the 
chain under which the FCLT is fulfilled. We also give a regime of the memory decay for which 
the limiting variance can be expressed in terms on the original process only. As applications, 
we derive FCLTs for autoregressive binary processes and Ising chains (Propositions 13.11 and 

mi)- 

The paper is organized as follows. In the rest of this section, we give some definitions 
and preliminaries. In Section [2l we state the main results. In Section [3] we introduce binary 
autoregressive processes and power-law Ising chains for which we give central limit theorems. 
Finally, Section S] is devoted to the proofs. 



1.2 Notation and preliminary definitions 

We consider a measurable space {E,£) where E is a finite alphabet and £ is the discrete 
(T-algebra. We denote {^,J-) the associated product measurable space with Q = E^. For 
each A C Z we denote i}\ = E^ and a\ for the restriction of a configuration cr G ^2 to 
namely the family {ai)i^A £ E^. Also, J^a will denote the sub-a-algebra of T generated by 
cylinders based on A (.^A-measurable functions are insensitive to configuration values outside 
A). When A is an interval, A = [k,n] with k,n £ 7^ such that k < n, we use the notation: 
(j^ = = LOk, ■ ■ ■ ,i^n, ~ ^[k,n] ^nd J^J^ = J^[k^n]- Foi' semi-intervals we denote also 

^<n = -^(-oo,n]i etc. The concatenation notation uj\aA, where A n A = 0, indicates the 
configuration on A U A coinciding with LVi for i £ A and with ai for i S A. 



1.3 Chains 

We start by briefly reviewing the well-known notions of chains in a shift-invariant setting. In 
this particular case, chains are also called 5- measures (see [16|). 

Definition 1.1 A (7-function g is a probability kernel g: Oq x $7Z^ [0,1], i-G., 

9{^0 I u;Zl) = 1, u;Zl G nz^. (1.1) 

The g-function g is: 

(i) Continuous if the function g (loq \ ■) is continuous for each ujq G Qq, i.e., for all e > 0, 
there exists n > so that 

lai^o I ^Zlo) - 9{c^o I 0-1^)1 < e (1-2) 
for all ^0 ^, cjO ^ G 0° ^ with uj\ = a\; 

(ii) Bounded away form zero if g {ujq \ • ) > c > for each loq G JIq;' 

(iii) Regular if f is continuous and bounded away from zero. 



Definition 1.2 A probability measure ¥ on (fi, J^) is said to be consistent with a g-function 
g if¥ is shift- invariant and 



h{uj)g{x\uj)F{duj)= I h{uj)¥{duj) (1.3) 

{uJO=x} 
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for all X £ E and T<:^i-measurahle function h. The family of these measures will he denoted 
by Q{g) and for each P G G{g), the process {Xi)i<zi on P) will he called a (7-chain. 

Remark 1.3 In the consistency definition \1.3\) . P needs only to he defined on {rL^_^, •^<o)- 
Because of its shift-invariance, P can be extended in a unique way to (fi, J^). That's why, 
without loss of generality, we can make no distinction between P on (Q^^, J-<o) and its 
natural extension on (il, J^). 

1.4 Regeneration 

The following regeneration result is due to Comets, Fernandez and Ferrari [8] (see Theorem 
4.1, Corollary 4.3 and Proposition 5.1). It will be the starting point of our analysis. 

Theorem 1.4 (Comets & al. (2002)) Let g be a regular g- function such that 




with the convention ctq =0. 
Then 

(i) there exists a unique probability measure P consistent with g; 

(a) there exists a shift-invariant renewal process (Ti)i^z with renewal distribution 

P(T,+i - T, > M) = pM, M > 0, i / (1.5) 

with pm the probability of return to the origin at epoch M of the Markov chain on NU{0} 
starting at time zero at the origin with transition probabilities 

'p{k, k + l) = Ofc, 

< p(/c,0) = l-afc, (1.6) 
^p{k,j) = otherwise 

and such that 

To < < Ti. (1.7) 

(Hi) the random blocks {{Xj : Ti < j < Ti^i)}i^z, where {Xi)i^z on {n,J^,¥) is the associated 
g-chain, are independent and, except for i = 0, identically distributed. 

(tv) 1 < E{T2 - Ti) = Pi < 

2 Main Results 

We are now ready to state the main results of this paper. 
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Theorem 2.1 (FCLT) Let {Xi)i^i he a regular g-chain satisfying the regeneration assump- 
tion JXTP and f : E ^ be a function such that 



Then, 



with 



E(/(Xo))=0. (2.1) 



1 1 """^ 

Sn:=^^f{Xi) — > AA(0,cj2) in distribution (2.2) 



"^-^= E(T.-T,) ^ ' (^-^^ 
Furthermore, i/E((T2 — Ti)^) < oo, t/ien 

a2 = E(/2(Xo)) +2J;e(/(Xo)/(X,)) < oo. (2.4) 

i>l 

Corollary 2.2 Lei {Xi)i^z be a regular g-chain such that there exist a G M, 6 G [l,oo), C > 
and K >2 such that for all k > K 

^ (log(fc))" 

afc>l-C' . (2.5) 

Let f: E satisfying / TO) . 

(^ij //a G M and 6 > 1, or a < —1 and b = 1, then ^2. 2\} is satisfied with cr^ defined by 112. 3\} . 
(a) //a G M and b > 2, or a < —1 and b = 2, then i2.2\) is satisfied with cr^ defined by 



n \ n 

j=0 



Remark 2.3 It is not easy to compare our results with previous ones based on mixing rates 
(see e.g. or U^ ]. because there is no general relationship between mixing rates and 

our continuity rates expressed in terms of {ak)k>o- However, in the case of regular g-chains 
with exponential continuity decay, the mixing rates are also of exponential decay (see [2^ or 
rO]/). and therefore CLT results as those describe in fL^ , Chapter 18, are fulfilled. 

3 Applications 

3.1 Binary autoregressive processes 

The binary version of autoregressive processes is mainly used in statistics and econometrics. It 
describes binary responses when covariates are historical values of the process (see McCuhagh 
and Nelder (1989), Section 4.3, for more details). 

In what follows, we consider the example that was introduced previously in [8j. For 
the alphabet E = {— 1,+1}, consider 9q a real number and {Ok : A; > 1) an absolutely 
summable real sequence. Let g : R ^ (0, 1) be a function strictly increasing and continuously 
differentiable. Assume that g{- \ ^j->zI^) is the Bernoulli law on {— 1,+1} with parameter 
9(^0 + Yjk>i ^k^^-k), that is, 

g{ + l\ojZl) =q(9o + Y,9kiv^k) = 1 - g{- 1 \ ivZl) . (3-1) 
^ k>i ^ 
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Denote 



n = ^ \0m\, 
m>k 



k>0. 



(3.2) 



Proposition 3.1 Let {Xi)i^i be a regular g-chain with g defined by i3. 1\) and 

/(x) = x-E(Xo). (3.3) 
(^) IfEk>orl<oo, then \g{g)\ = 1. 

(ii) If X]fc>o^*: ^ then the FCLT is satisfied with a"^ defined by ^2.3\) for f defined by 



(Hi) If there exists K > such that < C {\og{k)Y / , k > K, with C > 0, (a £ R and 
b > 2) or (a < —1 and b =2), then the FCLT is satisfied with cr^ defined by {2. 4^ for f 
defined by liS. 3\) . 

3.2 Power-law Ising chain 

For the usual Ising (Gibbs) model, the central limit theorem is well known (see Newman [20] ) . 
In the chain context, since the relationship between one dimensional Gibbs measures and 
chains, discussed in [lOj, does not allow to easily interpret Gibbs results in a chain setting, 
the problem becomes relevant. In what follows, we give central limit theorem for power-law 
Ising chains. 

For the alphabet E = {—1, 1}, consider the power-law Ising chain defined by 



exp 



g{uJo I uj-lo) 



fc=— oo 



E 



k=—oo 



(3.4) 



with 



1 

-/JjTj^'^OWA:, /C < > 0, p > 1. 



(3.5) 



Proposition 3.2 Let {Xi)i^z be a regular g-chain with g defined by ^3.4\\3. 5|] and 

fix) = X. (3.6) 

(t) If p> 3/2, then \G{g)\ = I. 

(ii) If p > 2, then the FCLT is satisfied with o"^ defined by 112. 3\) for f defined by ^3. 6]) . 
(Hi) If p > 3, then the FCLT is satisfied with o"^ defined by {2. 4^ for f defined by ^3. 0) . 
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4 Proofs 



4.1 Proof of Theorem 

Take (Ti)i^z, as given by Theorem 11.41 fii) and write 



n-l Ti-l 



n-1 



1=0 i=Ti «=7i(„) 



i=0 



with 



i(n) 



max{k > 1 : Tfc < n} if Ti < n, 
otherwise. 



(4.1) 



(4.2) 



Lemma 4.1 f/nder (T^, both u-^/^Y^JIq^ f{Xi) and n-^/"^ Y17=t,^,^^ fi^i) tend to zero 
probability. 



m 



Proof. For any X > 0, we obviously have 

Ti-l 



Therefore, 



with 



i=0 



sup P 

n>l ' 



> K^/^ ) < P ( Ti sup |/(x)| > K^/^ 



Ti-l 



1=0 



> i^Vn < ^(2^1 > Mk) 



K 



Mk = ^ and M = sup|/(x)|. 
On the other hand, by the shift-invariance of (Tj)^^^, 

n-l \ /^i{n)+i-l ^ 

/(X,) >KV^ <p ^ |/(x,)| >i^\A^ 



(n) 



< IP ( m(„)+i - T,(„)) sup I /(x) I > i^V^ 



where we used that i(0) = 0. Therefore, 



sup P 

n>l ' 



n— 1 



i=Ti 



¥(iTi-To)sup\fix)\> KV^) , 
V xeE J 



>K^ \ <¥{Ti-T^> Mk), 



(4.3) 

(4.4) 
(4.5) 



(4.6) 



(4.7) 



where Mk is defined by (14. 5p . Noticing that, under condition (|1.4p . both P(Ti > Mx) 
and P(ri — To > Mk) tend to zero as K goes to infinity, we can conclude that both 
n^^/"^ Z]2=o ^ fi-^i) and n^^^^ X]r=^r' ( j fi-^i) tend to zero in probability. I 
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Proposition 4.2 Let {Xi)i^z be a regular g-chain satisfying il-4\ ) cmd f : E 

following statements are equivalent: 

(i) Sn/y/n — > J\f{0,a'^) in distribution for 



Then the 



a 



E ( ( f{X^) 
HT2-T1) 



where Ti and T2 are defined in il.5\{T7T^ ; 
(a) {Sn/ \/n)n>o is bounded in probability; 

M IE(Er=T'/(^^)) =0 andE(^(E£T'/(^.))') < 00. 

Proof. The direction (i) ^ (ii) is trivial, so we only have to show (ii) =^ (iii) and (iii) =^ (i). 

To prove (ii)=^> (iii), we first remark that equation (|4.1|) . Lemma f4.1l and assertion (ii) imply 

that Yl'i=Ti ^ fi-^i) is bounded in probability. Then, by the converse of the central limit 

theorem for real i.i.d. sequences (see e.g. |18] . Section 10.1), we must have (iii). 

To prove (iii) =^ (i), we first see that 



i{n) 



n 



E(T2 - Ti) 



a.s., 



(4.9) 



which follows from Theorem 5.5.2 of [6] and the fact that (Tj_|_i — Ti)iyQ is an i.i.d. process. 
Let us denote 

6 = E ^""^ ^^^^ 

i=Tk 



n 



HT2-T1) 

Thanks to the Lemma HTTJ to prove (i), it is enough to show that 



(4.10) 



k=l 

This follows from the standard central limit theorem result 

ml) 



in law. 



(4.11) 



^ e(n) 



0, 



k=l 



E(r2 - Ti 



in law. 



and 



^ /i{n)-l e{n) \ 

V \ k=l k=l J 

To prove the latter, for any e > 0, let 



K' 



in probability. 



(4.12) 



(4.13) 



2(1 + E(a) 



(4.14) 



7 



First, remark that 



i(n) — 1 
k=l k=l 



<2P max 

\ i<nK' 



> Wne +P i n 



1 - e(n)| > nK'y 



Then, use the Kolmogorov's maximal inequahty, (iii) and ()4.14p . to get 



max 

j<nK' 



k=l 



\ 1 /"^ 

> V^e <^Var 



< -. 



, k=l 



Since 



Km P( U(n) - 1 - e(n)| > ni^' ) = 0, 
it follows from ([iJ5tiITT6]) that 



lim P 

n— >oo ' 



i(n) — 1 eC*^) 

E ^k-Yj^k 

k=l k=l 



> Jne < e. 



Proof of Theorem \2.1\ First, using the shift-invariance of P, we see that for all 

-I / / n— 1 \ / n— 2 n— 1 

n n 

\ \ «=u / 

„ n-l 



e(/2(Xo)) + - E(" - ^) IE(/(^o)/(X, 



1=1 



Then, under assumptions (jl.4p and ()2.ip . Theorem 1 of [2j insures that 

^e(/(Xo)/(X,)) <CJ]p, <oo. 



i>l i>l 

for some C > 0. Therefore, it follows from Kronecker's lemma that 

:<2 



n— >oo 77, 



e(/2(Xo)) +2J]e(/(Xo)/(XO) <oo. 



i>l 



In particular, Chebyshev's inequality implies that 



Sn 



n>l 



is bounded in probability. Then Proposition 14.21 concludes the proof of (j2.2ll2.3p 
To show ()2.4|) , we use first Proposition 14.21 (iii) to get 



E 



i{n)+l -| 2 

E 



/i(n)+l \ 

E( =E(e^)E(z(n) + l), 



where the rightmost equality follows from the Wald's Equation and the fact that i(n) + 1 is 
a stopping time w.r.t. {Ck)k>i- Since 



lim 



E{i{n)) 



1 



n— >oo n 



E(T2 - Ti) 



(see Theorem 5.5.2 in [6j), ([231) and (fOH]) give 



lim -E 

n— »oo n ' 



i{n)+l 
k=l 



But, we have 



lim f max£^ 1 = 

n^oo n \ k<n J 

(see [5] p. 90 for a proof) and therefore 

1 



lim-E(^|)=0, k e {i{n),i{n) + 1}. 

n— >oo n 



Hence, by ()4.25p and (I4.27|) . we finally have 



Now, if we show that 



and 



lim -E 

n— >oo n ' 



lim -E 



lim -E 

n— >oo n I 



i(n)-l 



fc=i 



Ti-1 

E/(^'= 

fc=l 



n-1 



E f^^f^) 



k=Ti 



0, 



i(n) 



(4.24) 



(4.25) 



(4.26) 



(4.27) 



(4.28) 



(4.29) 



(4.30) 



then, using Theorem 11.41 (iii), (j4.ip and (j4.28p . we can conclude the proof of ([27 

The proofs of (l4.29fH30]l are similar, we will prove ()4.30p only. To that aim, we first 
remark that 

A; P(ri - To > A;) ^ as k^oo, (4.31) 
because P(Ti — Tq > A;) is decreasing to and 



5^P(Ti - To > A:) = E(ri - To) = ^^-^ 



fe>0 



< oo, 



(4.32) 



where the rightmost equality uses that P(Ti — Tq = A;) = A;F(T2 — Ti = A:)/E(T2 — Ti) (see e.g. 
Lawler |T7j, Chapter 6, Section 6.2, equality (6.10)). Therefore, recalling (j4.5p and using the 
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shift-invariance of P, we have 

n-l 



1 



■E 



n 



lk=T, 



(n) 



n ^ ^ 



k=l 



(4.33) 



k=l 

< — V(2fc-i)p(ri-ro >fe), 



k=l 



which in view of (I4.3ip . goes to zero as n tends to infinity. I 
4.2 Proof of Corollary YI7^ 

Proof. To prove (i), it suffices to see that when {ak)k>o satisfies (|2.5p . then (a € M and h> 1) 
or (a < —1 and 6 = 1) if and only if Ylkyoi^ ~ ^k) < oo which is equivalent to nA:>o^fc ^ ^• 
Therefore, applying Theorem 12.11 first part, we get the result. 
To prove (ii), we first denote 

(log(A:))'^ 



afc = 1 - C ■ 



(4.34) 



and the associated pk, defined by the analogous of (II. 6p . Since 6 > 1, Proposition 5.5 (iv) in 
Fernandez, Ferrari and Galves [9] gives that there exists some constant Ci > so that 



PA: < Ci(l - afc), > 0. 

Therefore, using that at > at implies Pk < pk-, we have 

E((r2 - ri)2) =Y,kpk<CiY, k{i - dk) 



k>0 



k>0 



(4.35) 



(4.36) 



which is finite if and only if (a E M and & > 2) or (a < —1 and 6 = 2). Then, applying 
Theorem 12.11 second part, we get the result. I 



4.3 Proof of Proposition 13.11 

Proof. Define the variation by 

varfc =sup||5r(wo I t^lL) - 9{c^o I ^^-Dl - ^°oo>o-°oo ^ ^~oo^ ^-k = o--A:}> ^ > 0. (4.37) 
Then, because \E\ = 2, we have for any A; > (with the convention a^^ = 0) 



a, = inf {g{l \ uZ'^' aZl) +g{-l\ C'c^' aZl) : aZl G 



-1 , ,-fe-i t-fe-i 



1 - sup I - g{l 
1 — var^. 



LJ 



^ife) + 9{i I CL^' azl) : azl G ^zl ^z^-\ cL'' G nz'^'} 



(4.38) 
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Therefore, 



Ofc = 1 - sup |g ^^0 + kOj a-j + - q ^^o + J2 ^"i ~ ' ^-fc ^ ^-fcj • (^-39) 

Because q is continuously differentiable on a compact set, there exists C > such that 

varfc < Crfc, A; > 0. (4.40) 

To prove (i), it suffices to remark that 

var| < oo, (4.41) 

A:>0 

which is a tight uniqueness criteria in terms on the variation (see Johansson and Oberg [T3] 
and Berger, Hoffman and Sidoravicius [1]). 
To show (ii), we simply note that 

E(/(Xo)) = E(Xo - E(Xo)) = (4.42) 

and that 

^rfc<oo =^ J[ak>(^. (4.43) 

fe>0 fc>0 

Finally, to prove part (iii), it suffices first to combine (I4.38P and ()4.40p to get 

flfc > 1 - Crfe, A; > 0, (4.44) 
and then to apply Corollary 12.21 (ii) . 



4.4 Proof of Proposition 13.21 



Proof. We need the following well-known bound, whose proof we present for completeness is 
given in Appendix 1X1 (we follow the approach of |2H Lemma V.1.4]). 

Lemma 4.3 Let g he a g-function satisfying (fg.^D with X]fc=-oo I'Pkl < oo and h be a J-q- 
measurable function. Then, for any wZ^^, o"Z^ G ^-L' 



(4.45) 



ujoeE 



h{uJo)[g{^^o I (^^oo) - 9{^o I cT-i 



< sup \ h{x)\ sup 

x&E 



-1 



fc=— oo 

Applying the previous lemma for /i = 1, we obtain that, for any ti;<i_i, (T<j_i G 0<j_i, 

-1 J 



-1 1 

ai^o I ^-lo) -ai^o I cT-lo) < sup Y Tn^{^O^k-UJO(^k) 



k=—oo 



(4.46) 
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from which 

-k-i 

var,<2|/3| ^ < 2|/3|^, > 1, (4.47) 

j=-(X> l-^l 

is an immediate consequence. 

To prove (i), it suffices to see that (|4.47p with p > 3/2 imphes the vahdity of (|4.4ip . 

To prove (ii) and (iii), we first remark that similarly to the gibbsian setting, it can be 
easily checked that E(Xo) = and therefore under ()3.6p . ()2.ip is fulfilled. Then, we combine 
(OSI) and (OTP to get 

ak>l-m-^, k>l. (4.48) 
Thus, the results are direct consequences of Corollary 12.21 (i) and (ii). I 



A Appendix 



In this appendix we give the proof of Lemma 14. 3[ 

Proof. For all wZ^, crZ^ G fizl^ and < 6* < 1, define T^^^: E ^ {0, 1) by 



exp [oHUCo) + (1 








OHM + 


(1 - 9)HM 



with H^{i^)= M^-Uo)- (A.l) 



fc=— oo 



Then, to prove (|4.45p . it suffices to see that 



5]MCo)(5(Co|u;Z^)-5(Cok=i 



< 



d 

de 



de 



- E ^(^o)r^,.(Co) E {hM-hmYIA^^ 



voeE 



de 



< 



/ E (i^..(eo)-^a(eo)) - E (^-(^o)-^-(^o))r',^(^o) 



< \\h\\oosnp\HU^o)-H^{^o 
ioE 



(A.2) 
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